Feature Selection Guided by Structural Information

In generalized linear regression problems with an abundant number of features, lasso-type regularization which imposes an `-constraint on the regression coefficients has become a widely established technique. Crucial deficiencies of the lasso were unmasked when Zhou and Hastie (2005) introduced the elastic net. In this paper, we propose to extend the elastic net by admitting general nonnegative quadratic constraints as second form of regularization. The generalized ridge-type constraint will typically make use of the known association structure of features, e.g. by using temporalor spatial closeness. We study properties of the resulting ’structured elastic net’ regression estimation procedure, including basic asymptotics and the issue of model selection consistency. In this vein, we provide an analog to the so-called ’irrepresentable condition’ which holds for the lasso. An oracle property is established by incorporating a scaled `-constraint. Moreover, we outline algorithmic solutions for the structured elastic net within the generalized linear model family. The rationale and the performance of our approach is illustrated by means of simulated and real world data. keywords: generalized linear model, regularization, sparsity, p n, lasso, elastic net, random fields, consistency, epiconvergence, model selection, signal regression.